Inconsistencies in the highly cited publications indicator

Introduction The h-index, which was introduced by Hirsch (2005) has been criticized by Waltman and van Eck (2009, 2012) and Bouyssou and Marchant (2011) because it can lead to inconsistencies in the way in which scientists are ranked. This means that for the purpose for measuring the overall scientific impact of a scientist the h-index exhibits counterintuitive behavior. Waltman and van Eck (2012) discuss a large family of bibliometric indicators as alternatives to the h-index, but they pay special attention to the highly-citedpublications (HCPs) indicator. It is the purpose of the present investigation to show that the HCPs indicator suffers from similar inconsistencies. To avoid misunderstandings, I note that “the word ‘consistency’ has different meanings in different disciplines or even different areas of the same discipline” (Ye, 2012). In the present context the discussed inconsistencies refer to counterintuitive behavior of the h-index and the HCPs indicator in the ranking of individual scientists which occur due to performance improvement measured in terms of the numbers of papers or citations, respectively. In the following, I present three situations, in which the HCPs indicator behaves in a counterintuitive way, because the same performance improvement changes the ranking of individual scientists, or because the aggregation of data changes the ranking. It is assumed that the performance is measured in reference to a large dataset where the threshold of 10 citations defines which publications are counted as highly cited. The reference set should be large enough, so that the small changes in the example cases do not influence this threshold.